joshuark quotes a report from MIT Technology Review: It is impossible to make large language models fully secure against hacks because of a fundamental flaw in how they work, a team of researchers argue in a paper presented at the International Conference on Machine Learning, a top AI conference, this month. The claim has huge implications for the safety of this technology. By taking advantage of this flaw, which concerns how LLMs identify who or what is giving them instructions, the researchers were able to make popular LLMs spit out information they had been trained not to provide, such as how to synthesize cocaine and how to sabotage a commercial aircraft's navigation system. "There's a real probability that this is going to be a problem that's fundamentally unsolvable," says Charles Ye, an independent researcher and coauthor of the ICML paper. [...] The ICML paper describes attacks against several of OpenAI's models, but Cui and Ye say that they have since seen similar results with models made by Anthropic, Alibaba, and DeepSeek. Cui and her colleagues wanted to find out why an attack like chain-of-thought forgery was so effective. They suspected it had something to do with the mechanism that LLMs use to keep track of where their instructions are coming from. But what Cui and her colleagues discovered is that LLMs are in fact very bad at keeping track of different roles. In a series of experiments that looked at what was going on inside a handful of different models, the researchers found that LLMs seem to identify the role of a specific chunk of text not by the tags around it but by the style of that text and the words it contains. The upshot, the researchers claim, is that all an attacker needs to do to hack an LLM is write text that spoofs a certain role. And because roles are a fundamental part of how LLMs work, no amount of training will fully solve the problem. "There's going to be a huge economic incentive for people to do jailbreaks and prompt injections," says Cui. The best defense could be to expect the worst. Organizations shouldn't trust LLMs, and they should expect that anything done by agents could be unsafe, he says: "That's not a great solution, but it just might be what we have to do." "It's really incredible that these things are being deployed everywhere to control super-critical systems. There's been no study of the fundamental science here. We're all doing it ad hoc."

Read more of this story at Slashdot.

Security experts have been sounding the alarm for years about the risks of using generic TV boxes that promise unlimited content streaming for a one-time fee, warning that they secretly rent the user’s Internet connection out to strangers. But a groundbreaking new analysis finds these devices also routinely spoof themselves as mobile phones clicking ads on AI-generated websites as part of a sprawling operation that seeks to defraud online merchants and advertising networks.

Pedro Falé is a threat researcher with the security firm Bitsight. Falé told KrebsOnSecurity he was able to peer inside a vast and complex ad fraud network by registering an expired domain name that was used to coordinate fake ad clicks across a particularly popular brand of these streaming devices known as H96.

Falé said the domain he scooped up was previously used for telemetry, periodically collecting full hardware information and the entire list of installed apps from tens of thousands of H96 streaming sticks plugged into television sets around the globe. But upon inspecting the traffic being funneled to the domain, he discovered nearly all of the TV boxes transmitting data claimed to be mobile phone models from a variety of manufacturers, including Samsung, Vivo, Huawei, and Xiaomi.

“We noticed something was wildly wrong,” Falé said. “Multiple devices reporting to this factory Android TV Box backdoor were ‘phones.'”

The researcher found all of the devices reported having the same two apps installed, and that those apps were made by a company called Zhejiang Fengwo IoT Technology Ltd, an entity founded in 2019 in mainland China which operates an ad-publishing portfolio under the name Fengwo Group. Further investigation into the Fengwo Group revealed it has registered multiple patents that match the inner workings of these apps.

“Bitsight TRACE identified several Hong Kong, Singapore, and single person ‘legal’ shell identities used to collect the monetization and traced the operation back to a mainland China company known as Zhejiang Fengwo IoT Technology Co., Ltd, which operates under the Fengwo Group,” Falé wrote in a report released today about their findings.

Falé said an analysis of the apps shows they help to coordinate an ad fraud network that uses these H96 devices as a captive traffic source to click on ads at AI-generated websites operated by the Fengwo Group.

Bitsight discovered the websites contain machine-generated news articles and graphics across a range of categories, including finance, health, education, gaming, music and food blogs. But they also found none of those sites displayed ads unless the device visiting the page matched the spoofed mobile profile of these H96 devices.

AI DIGITAL HUMANS

The domain for the Fengwo Group — fwgcloud[.]com — claims the company is “redefining the boundaries of human-AI interaction,” and that it has created more than 120,000 “AI digital humans” available to rent for everything from emotional companionship to 24/7 customer service and creative design.

Falé said the Fengwo Group’s domain shared its SSL certificate data with other domains associated with the apps found on H96 devices, specifically the phone spoofing mechanism. He noted the domain also has an internal wiki platform that directly ties the Fengwo Group to a proprietary implementation of a Google-built visual programming language called Blockly, which was originally designed to help kids learn how to write software.

According to Bitsight, the Fengwo Group’s employees use Blockly to build the sham websites, allowing low-skilled operators to drag blocks of code together in their Blockly editor — without any need to understand what the underlying code blocks do or how they work.

“An operator can drag blocks together in their Blockly editor, to define each fraud routine, given a task type,” reads Bitsight’s report. “Once the routine is saved, it gets exported as JavaScript and uploaded to the S3 buckets. An operator doesn’t need as much understanding of the underlying technicalities, as it is all set in place for ease of use.”

Bitsight even found one of the Fengwo Group app developers mentioning exactly these advantages, noting the developer remarked that “only a small number of highly-skilled developers are needed to build the template execution-unit images,” and that “developers who create execution units from those templates have significantly lower technical requirements, greatly reducing the company’s operating costs.”

Falé said if a user’s H96 streaming stick is selected for a specific fraud task, it will be pushed the appropriate Blockly module according to the task desired, which can include silently launching a web browser, visiting websites, browsing pages, managing tabs, and clicking on ads.

To ensure the TV boxes masquerading as mobile phones can reliably click on ads displayed via the AI-generated websites, the Fengwo group “fuses three vision and reasoning systems into a single interface,” allowing the bots to correctly identify an ad on the webpage and navigate the site much like a human would, the Bitsight report observed.

TV ON? PROXY. TV OFF? AD FRAUD

Bitsight found the H96 devices were either relaying residential proxy traffic or participating in ad fraud, but never both at the same time. In fact, they concluded that when these TV boxes detect an HDMI signal from an attached television — indicating the user intends to stream video content — the box is usually functioning as a residential proxy. When the TV is off, it switches back to waiting for ad fraud jobs.

Falé said he believes the TV boxes are set up this way because its ad fraud activities are far more resource intensive and could interfere with the device’s stated purpose — streaming video content over the Internet.

Despite repeated warnings from the FBI and security industry leaders about the security and privacy risks of using these streaming devices, major e-commerce providers like Amazon, Best Buy, Newegg and others continue to sell hundreds of different models and brands that bundle unofficial versions of Google’s Android operating system and are frequently marketed (via online influencers) as a way to access a broad array of streaming services and live broadcasts without a subscription.

In addition to enlisting the user’s TV box in ad fraud networks, these off-brand streaming devices almost universally come with residential proxy software pre-installed. This software rents the user’s Internet address out to anonymous paying customers, who run the gamut from aggressive content scraping firms to ticket scalpers and outright cybercriminals.

What’s more, because these generic (and generally dirt cheap) TV boxes are all horribly insecure by default and bereft of any kind of authentication, installing one on your home or office network only invites further mischief. In January, the proxy tracking service Synthient documented how multiple botnets had rapidly enslaved millions of TV boxes using a complex interplay of security vulnerabilities in both the residential proxy software and the streaming devices themselves.

SHOW ME THE MONEY

Bitsight said it tracked approximately 38,000 TV boxes globally phoning home to the expired Fengwo Group domain, and based on that number the report estimates this ad fraud network brings in revenues of close to $50,000 a day (not counting substantial revenue from the residential proxy side of the business). However, Falé emphasized that these estimates are highly conservative and based on telemetry from just one of the Fengwo Group’s core (but older) domains.

As for the Fengwo Group’s claim to have 120,000 “digital humans” at their disposal, Bitsight’s report concludes it could be just a clever marketing scheme and/or a way to avoid drawing suspicion to the company’s operations.

“Historically, when dealing with proxy services or DDoS, we sometimes see these websites undertake inconspicuous facades, so as not to advertise their DDoS capability or botnet size,” Falé wrote in the report. “This could also be the case here.”

If the Fengwo Group truly does have tens of thousands of “AI humans” at its beck and call, it does not appear to have dedicated any of them to fielding inquiries from its own website. KrebsOnSecurity sought comment from the Fengwo Group by emailing the contact address listed on the company’s homepage, but the request bounced back with the reply, “Your message couldn’t be delivered to postmaster@fwgcloud[.]com. Their inbox is full, or it’s getting too much mail right now.”

As Bitsight’s analysis shows, when it comes to TV boxes and streaming sticks, it’s best to stick to name brands from reputable manufacturers, and then to be sparing and careful with any apps you choose to install on the device — as many of those can bundle residential proxy software as well. Google says consumers can confirm whether or not a device is built with the official Android TV OS and Play Protect certification by following these instructions.

Additionally, Synthient maintains a running list of IoT devices that have been known to ship to consumers with residential proxy software and other malicious apps pre-installed. Careful readers will notice Synthient’s list includes other IoT devices apart from streaming sticks and boxes: As the FBI has warned, residential proxy software has also been found in other popular consumer IoT devices from random brands, particularly digital photo frames.

Anthropic's Claude Mythos Preview has "found flaws in a weakened version of a digital encryption standard that is in pervasive use throughout the internet," reports The New York Times. Researchers said the model discovered novel attacks against weakened versions of AES and the experimental post-quantum HAWK system, including one that was 200 to 1,000 times faster than previous human-developed methods. From the report: The flaws identified do not concern a cryptographic standard currently in use today, which means that modern banking and communication systems are not subject to immediate potential intrusions from A.I. Instead, Anthropic's technology cracked a watered-down version of an algorithm for Advanced Encryption Standard, or A.E.S., a ubiquitous protocol that safeguards web traffic, wireless networks, data storage and more. It is common to perform tests on weaker versions of encryption algorithms to understand whether more powerful computers could someday crack the actual standards, akin to solving a simpler math problem to identify whether patterns may exist for a more complicated one. In the testing, Mythos was able to break the weaker version of Advanced Encryption Standard in a way that Anthropic said made an assault 200 to 1,000 times faster than what previous human research had managed to do. While the immediate ramifications are minimal, the long-term implications could be significant. In previous tests, large-language models seemingly could not match or best what humans can do in the mathematically dense field of cryptographic research, but their rapid advances could suggest a future in which top models can surmount traditional internet security protections that are foundational to just about everything that takes place on the internet. [...] In addition to the attack on the encryption standard, Mythos also orchestrated another improved attack against a different digital cryptographic system known as HAWK that is designed to be bulletproof against both traditional and quantum computers. HAWK is not currently in use, but under consideration by the National Institute of Standards and Technology to become a new standard. The HAWK attack was validated by its authors, and independent cryptographers reviewed the Advanced Encryption Standard attack, Anthropic said, adding that it had shared its findings with the U.S. government and industry partners ahead of publication. Mythos devised the cryptographic attack on A.E.S. almost entirely autonomously, Anthropic said, but only after first refusing to contemplate the problem because it believed it was impossible to improve on existing methods of analysis. But after some coaxing, the chatbot sat with the puzzle for about a week before engineering its novel attack. Two human researchers then worked for nearly a month to verify that the method appeared correct. "Given that we are constantly underestimating the power and time of availability of future models, are we really comfortable that two years from now strong encryption won't be threatened?" said Glenn S. Gerstell, the former general counsel at the National Security Agency. "Mathematicians would tell you that it shouldn't be possible given current computing powers to break strong encryption in any meaningful time," added Mr. Gerstell, who helped write a report on cryptology in 2022. "But I don't think the capabilities of future models in the medium term -- before quantum computing or quantum-proof cryptography -- should be dismissed as trivial in this context."

Read more of this story at Slashdot.

eBay and several former executives have agreed to pay $56 million to Ina and David Steiner, the newsletter writers targeted in a 2019 corporate harassment campaign that involved threats, surveillance attempts, and deliveries of live insects and other disturbing items. The settlement closes the couple's civil case after seven former employees pleaded guilty to criminal charges related to the scheme. TechCrunch reports: Ina and David Steiner, a married couple and the co-authors of EcommerceBytes, inspired the ire of high-level eBay executives after occasionally criticizing the company in their newsletter. In 2019, a plot was concocted to intimidate the couple into halting their negative coverage. Executives used sock puppet social media accounts to harass the couple, while also sending them anonymous threatening letters and bizarre items in the mail -- including live spiders and cockroaches, pornographic magazines, a bloody pig mask, a funereal wreath, and a book about surviving the death of a spouse. According to previously released court documents, a plan that was attempted but never successfully carried out involved affixing a GPS tracking device to the couple's car. Yet another internally broached plan involved sending a "Samoan gang" to the Steiners' home. The settlement this week resolves a 2021 civil case brought by the couple against eBay. The law office representing the Steiners writes that the settlement includes $46.15 million paid to the couple by eBay itself, as well as $2 million from former eBay executive CEO Devin Wenig. Additionally, $500,000 will be paid out to the couple from former eBay executive Wendy Jones, as well as $50,000 from former eBay executive Steve Wymer. Additional funds are being paid to various non-profits. In 2022, seven former eBay employees were criminally charged and pled guilty in relation to the plot, including the company's former security chief, James Baugh -- who was sentenced to nearly five years in prison. Others indicted by the U.S. Department of Justice include David Harville, Brian Gilbert, Stephanie Popp, Stephanie Stockwell, Philip Cooke, and former eBay contractor Veronica Zea.

Read more of this story at Slashdot.

Ardent anti-vaccine activist and current US Health Secretary Robert F. Kennedy Jr. is reportedly on thin ice with Trump for not being anti-vaccine enough for the president's liking, according to a report late Monday from The Wall Street Journal.

Trump has become fixated on the completely false, thoroughly debunked claim that vaccines cause autism, the Journal reported. People familiar with the president's scattered thinking say Trump wants an autism remedy to be part of his legacy and thinks scaling back childhood vaccinations is a way to secure that achievement. As such, he has grown frustrated and disappointed that Kennedy hasn't done more to cut back federal childhood vaccine recommendations or confirm a link to the neurodevelopmental disorder.

In May, the president reportedly aired his disappointment, calling out Kennedy for having "the yips" that were preventing him from acting on his zealous anti-vaccine agenda while holding the highest health position in the country. In June, Trump reportedly vented in the Oval Office to Kennedy that he wasn't doing enough.

Kennedy has reportedly been taken aback by the criticism—and it's likely that outside public health and medical experts would have been, too, if they were present. To experts, Kennedy has been diligently working to undermine life-saving vaccines, endangering public health in the process.

Some notable examples include Kennedy's overhaul of a key vaccine advisory committee, which he stacked with like-minded anti-vaccine allies. Those allies then arbitrarily ditched evidence-based recommendations for the birth dose of the hepatitis B vaccine and a combination shot against measles, mumps, rubella, and varicella (chickenpox) called MMRV. Kennedy also single-handedly trimmed the US childhood vaccine schedule to resemble that of Denmark's, a small country with universal healthcare. He directed the Centers for Disease Control and Prevention to stop promoting seasonal flu vaccines with prepaid advertisements during a particularly deadly flu season. And he edited a CDC website to falsely link vaccines to autism.

Political risks

Some of Kennedy's efforts to trash vaccines have been blocked in court—namely, the changes to the advisory committee and the childhood vaccine schedule. But Kennedy is now laying the regulatory groundwork to sabotage vaccination rates in ways that can't be undone so easily.

Kennedy's anti-vaccine meddling in federal policy has been so apparent that polling in December found that Kennedy's efforts to cut vaccines were broadly unpopular and "politically risky." As such, White House political advisors have urged Trump and Kennedy to tone down the anti-vaccine rhetoric, which Kennedy appeared to be doing—publicly at least. In recent public appearances, Kennedy has avoided the topic of vaccines.

But it seems Trump has not aligned with his advisors and still wants Kennedy to demolish vaccination policy with wild abandon, despite the health and political dangers. With Kennedy apparently not working fast enough, the Journal reported speculation that Kennedy could be sidelined or replaced. Already, Kennedy’s Medicare chief, Chris Klomp, has been "empowered to run every corner" of the Department of Health and Human Services on behalf of the Secretary, the paper reported. There are also rumors that Mehmet Oz, the Centers for Medicare and Medicaid Services administrator, could become health secretary after the midterm elections.

Read full article

Comments

Anti-vaccine Health Secretary Robert F. Kennedy Jr. has made it one of his top priorities to convince Americans to "eat real food," including more fresh fruits and vegetables. But, it's a hard sell when those real foods are causing real eruptions of explosive diarrhea across the country.

The Food and Drug Administration is investigating six outbreaks of foodborne diarrheal illnesses caused by Cyclospora, a unicellular parasite known for spreading from human feces onto fresh produce. Past outbreaks have been linked to leafy greens, fresh basil, fresh cilantro, snow peas, green onions, and raspberries. The Centers for Disease Control and Prevention has tallied a record high of over 11,500 confirmed and probable cases across 41 states, with over 300 requiring hospitalization. And that case total is almost certainly an undercount.

The largest of the outbreaks is linked to Taylor Farms' iceberg lettuce that was sold at Taco Bell restaurants and many other companies. The outbreak has sickened thousands of people across nine states, up from five states identified last week. One of the affected states is Michigan, which alone has reported 9,253 cases as of July 27. The other five outbreaks have no identified source.

Given the situation, it's no surprise that a CBS News poll over the weekend found that about 40 percent of Americans are now buying or eating less produce amid the outbreak. Only 4 percent reported they were following Kennedy's guidance of eating more.

The finding appears to be a setback to Kennedy's Make America Healthy Again agenda, which has typically focused on overhauling nutrition guidance while neglecting food safety and infectious diseases.

Under Kennedy's leadership, the FDA and CDC have both suffered draconian budget cuts and been drained of experienced career staff who know best how to identify, respond to, and effectively inform the public about such outbreaks. The CDC has lost about 25 percent of its staff, while FDA is down about 20 percent, CBS has reported.

Last year, the FDA also announced that it was delaying a new rule on food traceability, which would have made it easier for FDA investigators to identify and remove from the market foods linked to illnesses, such as Taylor Farms' lettuce.

"Catastrophic" mismanagement

Last week, Kennedy claimed that the Cyclospora surge was "under control," but critics and former FDA officials have scoffed at the statement.

"If history is any judge, when the Trump administration declares a crisis 'under control', it’s time to brace for the worst yet to come," Brad Woodhouse, president of the public health nonprofit Protect Our Care, said in a statement.

Frank Yiannas, who served as FDA's deputy commissioner for food policy and response in the first Trump administration and under Biden appeared equally skeptical. Yiannas has publicly called for an independent investigation into the federal government's handling of the Cyclospora outbreaks.

"This one deserves a lot of questions, and it is starting to approach a catastrophic level in terms of how mismanaged it’s been on multiple fronts," Yiannas told Politico. As examples of the mismanagement, Yiannas noted the outbreak's extreme size—already well over double the typical number of cases seen in recent years—as well as mixed messaging and confusion among consumers. He also said there was a "lack of strong federal leadership."

Susan Mayne, who served as director of the FDA’s Center for Food Safety and Applied Nutrition from 2015 to 2023, said the explosive outbreak was a "wake-up call" on the interconnectedness of food safety and nutrition. In an op-ed published in Stat, she noted seeing people online turning to processed foods that they now considered safer than salads and produce—such as pizza, bagged chips, and store-bought cookies—foods Kennedy has tried to steer Americans away from.

But "Oreos never gave me explosive diarrhea!" one person said, according to Mayne.

"Consumers should not be faced with a trade-off between nutrition and food safety—health is made possible when food safety supports consumer confidence in fruits and vegetables," Mayne wrote. “‘Real foods' are vulnerable to real pathogens."

Read full article

Comments